Studying Rational Agency and Epistemic Communities with Large Language Models:
Review, How-To, and Reflection

Workshop on Computational Models in Social Epistemology
Bochum, Dec 6-8 2023
https://github.com/debatelab/genai-epistemology

Gregor Betz (DebateLab@KIT)

Review

Autonomous LLM-based in Agents

📄 Wang et al. (2023)

Artificial Deliberating Agents

📄 Betz (2022)

“Debating” LLMs

📄 Du et al. (2023)

“Debating” LLMs

Prompt: “These are the solutions to the problem from other agents: <other agent responses> Using the reasoning from other agents as additional advice, can you give an updated answer? Examine your solution and that other agents. Put your answer in the form (X) at the end of your response.”

📄 Du et al. (2023)

How-To

Code Boilerplate

https://github.com/debatelab/genai-epistemology

Skeleton Bounded Confidence Agent

class AbstractBCAgent():

    def update(self, community):
        opinions = [peer.opinion for peer in self.peers(community)]
        self.opinion = self.revise(opinions)

    def peers(self, community):
        peers = [
            agent for agent in community
            if self.distance(agent.opinion) <= epsilon
        ]
        return peers

    def distance(self, opinion):
        pass 

    def revise(self, opinions):
        pass

Numerical BC Agent

class NumericalBCAgent(AbstractBCAgent):

    def distance(self, opinion):
        """calculates distance between agent's and other opinion"""
        return abs(opinion - self.opinion)

    def revise(self, opinions):
        """revision through weighted opinion averaging"""
        alpha = self._parameters.get("alpha", .5)
        revision = alpha * self.opinion + (1-alpha) * np.mean(opinions)
        return revision

Numerical BC Model: Results

Numerical BC Agent

class NumericalBCAgent(AbstractBCAgent):

    def distance(self, opinion):
        """calculates distance between agent's and other opinion"""
        return abs(opinion - self.opinion)

    def revise(self, opinions):
        """revision through weighted opinion averaging"""
        alpha = self._parameters.get("alpha", .5)
        revision = alpha * self.opinion + (1-alpha) * np.mean(opinions)
        return revision

Large Language Model

model = lmql.model(
    "local:HuggingFaceH4/zephyr-7b-alpha",
    device_map = "auto",
    load_in_8bit=True,
    low_cpu_mem_usage=True
)

Agreement Prompt

Revision Prompt

Natural Language BC Agent

class NaturalLanguageBCAgent(AbstractBCAgent):

    def distance(self, other):
        """distance as expected agreement level"""
        lmql_result = agreement_lmq(
            self.opinion, other, **kwargs
        )
        probs = lmql_result.variables.get("P(LABEL)")
        return sum([i*v for i, (_, v) in enumerate(probs)])/4.0

    def revise(self, peer_opinions):
        """natural language opinion revision"""
        revision = revise_lmq(
            self.opinion, peer_opinions, **kwargs
        )
        return revision

Natural Language BC Model: Results

alpha=“very high”; epsilon=0.40.5; topic=“veganism”

Reflection

Why LLM-based ABMs?

  1. Re-create and probe our epistemological (computational) models.
  2. Simulate and test scientific methodologies, reasoning modes, principles of rationality.
    Any! (E.g. value-free ideal.)
    Without formalizing them.

🤔 Are LLMs suited for building epistemic agents?

🤔 LLMs’ abilities: Reasoning? (1/3)

📄 Pan et al. (2023)

🤔 LLMs’ abilities: Reasoning? (2/3)

📄 Morris et al. (2023)

🤔 LLMs’ abilities: Reasoning? (3/3)

📄 AI4Science and Quantum (2023)

🤔 LLMs’ abilities: Beliefs?

📄 Betz and Richardson (2023)

🤔 LLMs’ abilities: Unhuman?

But humans’ cognitive architecture is fundamentally different from LLMs’ , or is it?

📄 Goldstein et al. (2020)

🤔 LLMs’ abilities: Unhuman?

📄 The neural architecture of language: Integrative modeling converges on predictive processing. (Schrimpf et al. 2021)

TLDR It is found that the most powerful “transformer” models predict nearly 100% of explainable variance in neural responses to sentences and generalize across different datasets and imaging modalities […].

📄 Brains and algorithms partially converge in natural language processing. (Caucheteux and King 2022)

TLDR This study shows that modern language algorithms partially converge towards brain-like solutions, and thus delineates a promising path to unravel the foundations of natural language processing.

📄 Mapping Brains with Language Models: A Survey. (Karamolegkou, Abdou, and Søgaard 2023)

ABSTRACT […] We also find that the accumulated evidence, for now, remains ambiguous, but correlations with model size and quality provide grounds for cautious optimism.

📄 Artificial neural network language models predict human brain responses to language even after a developmentally realistic amount of training. (Hosseini et al. 2022)

TLDR [A] developmentally realistic amount of training may suffice and […] models that have received enough training to achieve sufficiently high next-word prediction performance also acquire representations of sentences that are predictive of human fMRI responses.

Conclusion

LLMs suited for building epistemic agents?

  • ✅ reasoning skill
  • ✅ consistent belief states (syn- and diachronic)
  • ✅ similarities to human cognition

Come, join the party! 🎉

Conclusion

Vanishing distinctions (due to AGI):

  • simulating science vs doing science
  • epistemology vs AI

Conclusion

Epistemic redundancy (due to AGI) brings profound philosophical challenges:

  • What role for humans in science? University?
  • Science in a democracy? AGI-proof well-ordered science?
  • If AGI is science’s ultimate “Freudian” offence and blow to humans’ collective narcissism, which revisions of our self-conception (as rational & moral persons) may avoid existentialist disaster? AGI-proof humanism?

Backup

References

AI4Science, Microsoft Research, and Microsoft Azure Quantum. 2023. “The Impact of Large Language Models on Scientific Discovery: A Preliminary Study Using GPT-4.” https://arxiv.org/abs/2311.07361.
Betz, Gregor. 2022. “Natural-Language Multi-Agent Simulations of Argumentative Opinion Dynamics.” Journal of Artificial Societies and Social Simulation 25 (1): 2. https://doi.org/10.18564/jasss.4725.
Betz, Gregor, and Kyle Richardson. 2023. “Probabilistic Coherence, Logical Consistency, and Bayesian Learning: Neural Language Models as Epistemic Agents.” PLOS ONE 18 (2): 1–29. https://doi.org/10.1371/journal.pone.0281372.
Caucheteux, Charlotte, and Jean-Rémi King. 2022. “Brains and Algorithms Partially Converge in Natural Language Processing.” Communications Biology 5 (1): 134.
Du, Yilun, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum, and Igor Mordatch. 2023. “Improving Factuality and Reasoning in Language Models Through Multiagent Debate.” https://arxiv.org/abs/2305.14325.
Goldstein, Ariel, Zaid Zada, Eliav Buchnik, Mariano Schain, Amy Rose Price, Bobbi Aubrey, Samuel A. Nastase, et al. 2020. “Thinking Ahead: Spontaneous Prediction in Context as a Keystone of Language in Humans and Machines.” bioRxiv. https://api.semanticscholar.org/CorpusID:227915127.
Hosseini, Eghbal A, Martin Schrimpf, Yian Zhang, Samuel Bowman, Noga Zaslavsky, and Evelina Fedorenko. 2022. “Artificial Neural Network Language Models Align Neurally and Behaviorally with Humans Even After a Developmentally Realistic Amount of Training.” BioRxiv, 2022–10.
Karamolegkou, Antonia, Mostafa Abdou, and Anders Søgaard. 2023. “Mapping Brains with Language Models: A Survey.” arXiv Preprint arXiv:2306.05126.
Morris, Meredith Ringel, Jascha Sohl-dickstein, Noah Fiedel, Tris Warkentin, Allan Dafoe, Aleksandra Faust, Clement Farabet, and Shane Legg. 2023. “Levels of AGI: Operationalizing Progress on the Path to AGI.” https://arxiv.org/abs/2311.02462.
Pan, Liangming, Michael Saxon, Wenda Xu, Deepak Nathani, Xinyi Wang, and William Yang Wang. 2023. “Automatically Correcting Large Language Models: Surveying the Landscape of Diverse Self-Correction Strategies.” https://arxiv.org/abs/2308.03188.
Schrimpf, Martin, Idan Asher Blank, Greta Tuckute, Carina Kauf, Eghbal A. Hosseini, Nancy Kanwisher, Joshua B. Tenenbaum, and Evelina Fedorenko. 2021. “The Neural Architecture of Language: Integrative Modeling Converges on Predictive Processing.” Proceedings of the National Academy of Sciences 118 (45): e2105646118. https://doi.org/10.1073/pnas.2105646118.
Wang, Lei, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, et al. 2023. “A Survey on Large Language Model Based Autonomous Agents.” https://arxiv.org/abs/2308.11432.